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Purpose: The aim of this study is to determine the perceived familiarity of medical residents with statistical 
concepts, assess their ability to integrate these concepts in clinical scenarios, and investigate their suscepti- 
bility to the gambler's fallacy and the conjunction fallacy. 

Methods: A multi-institutional, cross-sectional survey of Greek medical residents was performed. Participants 
were asked to indicate their familiarity with basic statistical concepts and answer clinically oriented questions 
designed to assess their biostatistics knowledge and cognitive biases. Univariate, bivariate, and multivariate 
statistical models were used for the evaluation of data. 

Results: Out of 153 respondents (76.5% response rate), only two participants (1.3%) were able to answer 
all seven biostatistics knowledge questions correctly while 29 residents (19%) gave incorrect answers to all 
questions. The proportion of correct answers to each biostatistics knowledge question ranged from 15 to 
51.6%. Residents with greater self-reported familiarity were more likely to perform better on the respective 
knowledge question (all p <0.01). Multivariate analysis of the effect of individual resident characteristics on 
questionnaire performance showed that previous education outside Greece, primarily during medical school, 
was associated with lower biostatistics knowledge scores (p < 0.001). A little more than half of the respon- 
dents (54.2%) answered the gambler's fallacy quiz correctly. Residents with higher performance on the 
biostatistics knowledge questions were less prone to the gambler's fallacy (odds ratio 1.38, 95% confidence 
intervals 1.12-1.70, p =0.003). Only 48 residents (31.4%) did not violate the conjunction rule. 
Conclusions: A large number of medical residents are unable to correctly interpret crucial statistical concepts 
that are commonly found in the medical literature. They are also especially prone to the gambler's fallacy 
bias, which may undermine clinical judgment and medical decision making. Formalized systematic teaching 
of biostatistics during residency will be required to de-bias residents and ensure that they are proficient in 
understanding and communicating statistical information. 
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Introduction 

Biostatistical literacy is imperative for physicians in train- 
ing who must keep abreast of current medical knowledge 
and be able to communicate statistical and epidemiologi- 
cal information to patients and colleagues. Furthermore, 
medical residents may be particularly vulnerable to the 



suggestions of the published literature and other more 
questionable sources such as promotional brochures and 
websites (1, 2). The vast majority of pubhshed medical 
research contains at least some elementary statistical ana- 
lysis, and it has consistently been shown that errors in 
such data are frequently found, even in well-respected 
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textbooks (3-7). It is therefore crucial for residents to 
possess the necessary basic biostatistical knowledge to 
critically evaluate and apply original research data. 

Previous surveys conducted in the 1980s and early 1990s 
demonstrated that physicians have poor understanding 
of simple biostatistical concepts (8-11). Physicians today 
may have even more pronounced difficulties in compre- 
hending and integrating data derived from the increasingly 
sophisticated statistical methodologies used in contem- 
porary medical literature (7, 12-14). More recent studies 
have shown that internal medicine, emergency medicine, 
and obstetrics-gynecology residents have considerable 
difficulties in interpreting statistical results found in pub- 
lished clinical research (15-17). Therefore, despite the fact 
that many medical schools include statistical courses in 
their curricula, it should not be taken for granted that 
residents can effectively assess biostatistical information 
and use it to the best advantage of their patients. 

A number of predictable biases in human information 
processing may considerably hinder sound clinical deci- 
sion making (18-21). Therefore, documenting and eluci- 
dating these processes may improve patient outcomes, 
prevent the inappropriate utilization of medical resources, 
and reduce health care expenditures. A previous instru- 
ment designed to evaluate such cognitive biases among 
physicians (22) was recently reported to have very limited 
measurement properties (23). The representativeness heu- 
ristic is a known factor that may lead to bias in probability 
estimation by physicians (24, 25). The gambler's fallacy 
is a classic cognitive bias produced by the representative- 
ness heuristic and arises when a person assumes that a 
deviation from what occurs on average or in the long term 
will be corrected in the short term (26). A common ex- 
ample is when tails are repeatedly obtained while tossing a 
fair coin leading the gambler to incorrectly expect that 
heads are 'due' for the next toss. In reality, the previous 
tosses have no bearing on the outcome of future tosses. 
The gambler's fallacy has been shown to be by far the most 
frequent bias arising in this probability judgment situa- 
tion, particularly among highly educated individuals (27, 
28). The conjunction fallacy is another cognitive illusion 
of probability judgment which was first documented using 
the well-known 'Linda' example whereby the stereotypical 
description of Linda as a social activist compels respon- 
dents to rank the probability that she is a feminist bank 
teller as greater than the probability that she is a bank 
teller. However, this response is considered erroneous as 
feminist bank tellers cannot be more frequent or more 
probable than the more inclusive class of bank tellers (29). 

The aim of the present multicenter, cross-sectional 
study was to determine the perceived familiarity of medi- 
cal residents with statistical concepts and assess their 
ability to integrate these concepts in actual clinical sce- 
narios. Furthermore, the incidence of the gambler's fallacy 
and the conjunction fallacy among medical residents were 



investigated using clinical vignettes that were designed to 
tap these two domains with specific reference to medicine. 
The effects of individual resident characteristics such as 
age, gender, residency year, and past biostatistics training 
on the cognitive biases and biostatistics knowledge of 
medical residents were also evaluated. 

Methods 

Participants and data collection 

The reporting of this cross-sectional study conforms to 
the STROBE statement (30). The data presented here are 
part of a large multi-institutional project evaluating the 
biostatistical competency and cognitive biases of medical 
residents. Respondents were originally recruited to parti- 
cipate in a cross-sectional randomized experimental study 
of medical resident's Bayesian reasoning performance and 
the sample size was determined according to that study's 
requirements. The survey was conducted from January to 
March 2010, and the participants were chosen from eight 
major Greek hospitals in the city of Athens (Table 1). The 
Greek system does not allow official residency training in 
private hospitals and therefore all institutions surveyed in 
this study were public. Every resident was given a number 
according to the official enrolment lists that had been 
provided by each of the hospitals. Out of 1 ,272 eligible 
residents in all hospitals, 200 (25 from each hospital) were 
randomly selected by a computerized method to partici- 
pate in the sample, a number that represents approxi- 
mately 2% of the total number of residents training in 
Greece (31). Participation was elective and all participants 
were approached during breaks from work or training, 
were informed that responses would be anonymous 
and were blinded to the scope and purpose of the study. 
The residents were asked to return the completed ques- 
tionnaires to a sealed box provided in each hospital. The 
study received permission from the authorized personnel 
at each institution (i.e., the Department or Scientific 
Meeting Chair). The study complied with Greek require- 
ments for survey studies, and ethical approval was not 
required as responses were fully anonymous, participa- 
tion was elective, all participants were approached during 
breaks from work or training, and the study did not con- 
tain questions on sensitive topics. Participants' informed 
consent was indicated by each individual's willingness to 
complete and return the questionnaire. 

Survey measures 

The full questionnaire was developed in the Greek 
language and is available from the corresponding author. 
The first seven questions queried the socio-demographic 
profile, specialty choices, previous education outside 
Greece, current training level, residency year, and past 
training in biostatistics of residents. We combined resi- 
dencies according to their conceptual and occupational 
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Table 1. Resident demographic, educational, and residency profile 



1. Age 


Median: 32 years 2. Gender^ 
Range: 24—44 years 


Male 
112 (73.2%) 


Female 
41 (26.8%) 


3. Year of 
residency 


Median: 
Range: 1 


2 years 
—7 years 


4. Hospital^ '' 


1st I.K.A. 251 G.N.A. Aglaia Kyriakou 


Alexandra 


Asklipiio 


Elpis 


Evangelismos 


Metaxa 




Hospital Hospital Hospital 


Hospital 


Hospital 


Hospital 


Hospital 


Hospital 




18 17 21 


21 


21 


19 


15 


21 




(11.8%) (11.1%) 13.7%) 


(13.7%) 


(13.7%) 


(12.4%) 


(9.8%) 


(13.7%) 


5. Residency 




Surgical 


Diagnostic and laboratory 







field^ Medical specialties specialties specialties 

71(46.4%) 67(43.8%) 15(9.8%) 



6. Other 
graduate 
degrees^'*^ 



No 

136 (88.9%) 



PhD 

6 (3.9%) 



MSG 

14 (9.2%) 



7. Residency 
program type^ 



University 

based 
28 (18.3%) 



Community 

based 
125 (81.7%) 



8. Education 
abroad^" 


No 

96 (62.7%) 


Primary or 
secondary 
education 
2 (1 .3%) 


Medical School 
45 (29.4%) 


Prior to 
beginning 
residency 
4 (2.6%) 


During 
Residency 
10 (6.5%) 


Other 
3 (2%) 


9. Biostatistics 
training^'^ 


No 

28 (18.3%) 


Primary or 
secondary 
education 
7 (4.6%) 


Medical School 
119 (77.8%) 


Prior to 
beginning 
residency 
0 


During 
Residency 
6 (3.9%) 


Other 
11 (7.2%) 



^Percentages in parentheses represent the rate percent to the study's 153 respondents. 
''Percentages in parentheses do not add up to 100% due to rounding. 

''Percentages in parentheses do not add up to 100% due to rounding and overlapping of degrees (three respondents had both PhD and 
MSc degrees). 

''Percentages in parentheses do not add up to 100% due to rounding and overlapping of foreign educational experiences. 
''Percentages in parentheses do not add up to 100% due to rounding and overlapping of past biostatistics training settings. 



relations and formed three different medical 'fields': 1) 
internal medicine (« =71; dermatology, pediatrics, neurol- 
ogy, and general practice were also included in this group); 
2) surgical specialties (n = 67); and 3) diagnostic and 
laboratory specialties (« = 15). The participants were then 
asked to indicate their understanding of the statistical 
concepts of standard deviation (SD), standard error of 
the mean (SEM), /^-values, confidence intervals (CI), cor- 
relation coefficients, relative risk (RR), sensitivity, and 
positive predictive value (PPV) by rating their famil- 
iarity with each concept on a 7-point Likert scale ranging 
from 'none' (score of 1) to 'excellent' (score of 7). The next 
set of questions was developed by one of the authors and 
assessed the biostatistics knowledge (Table 2) and cogni- 
tive biases (Table 3) of medical residents. 

A mathematician from the Section of Statistics and 
Operations Research of the Department of Mathematics 
of the University of Athens reviewed the questions for 
clarity and mathematical appropriateness. In order to 
assess the intelligibility, interpretation, content validity, 
adequacy of response options, and clinical relevance of the 



vignettes, the questionnaire was pretested in a separate 
group of seven medical residents, including two residents 
with advanced training in epidemiology and biostati- 
stics. The residents completed the questionnaire and 
provided oral feedback which resulted in the removal of 
one question to avoid duplication of similar concepts. 
Both residents with advanced biostatistics backgrounds 
answered all questions correctly. Of the remaining five 
residents, four answered four of seven biostatistics knowl- 
edge questions correctly while one resident correctly 
answered two out of seven questions. Four of these five 
residents selected the 'approximately lO'/o' choice in the 
gambler's fallacy scenario and answered that the prob- 
ability of suffering from coronary artery disease (CAD) 
and Huntington's disease is greater than the probability of 
suffering from Huntington's disease. Despite extensive 
discussion of the conjunction fallacy vignette's probabil- 
istic nature, these residents were very reluctant to accept 
the strictly mathematical reading of the problem. 

A 'total biostatistics knowledge' score, which equally 
weighted each of the seven questions listed in Table 2, was 
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Table 2. Biostatistics knowledge questions 



Response 





Scenario description 




Multiple choices (select only one) 


frequencies'* (%) 


SD vignette 


You read in a medical paper that 1 00 patients had 


* 


It is 3pproxim3tely 95% certain that the 


80 


(52.3) 




fasting serum glucose levels of 153 mg/dl + 6 mg/dl 




true mean value lies between 147 and 








(mean + SD). Which of the following statements is the 




Toy mg/ui 








most correct? 


* 


Approximately 50% of patients had 


12 


(7.8) 








fasting serum glucose 153 mg/dl 










* 


Approximately 95% of patients had 


49 


(32)'' 








fasting serum glucose 141—165 mg/dl 












1 do not know and do not wish to guess 


12 


(7 H\ 


SEM vignette 


You read in a medical paper that 2 hours following 


* 


It is approximately 95% certain that the 


35 


{22.9f 




administration of a drug the mean systolic pressure of 




true mean value lies between 122 and 








the patients was 138 mm Hg + 8 mm Hg 




154 mm Hg 








(mean + SEM). Which of the following statements is 


* 


Approximately 50% of patients had mean 


11 


(7.2) 




the most correct? 




systolic pressure 138 mm Hg 










• 


Approximately 95% of patients had mean 


85 


(55.6) 








systolic pressure 130—146 mm Hg 










• 


1 do not know and do not wish to guess 


22 


(14.4) 


p-value 


You read in a clinical trial report that the treatment 


• 


The treatment is definitely better 


58 


(37.9) 


vignette 


under investigation showed statistically significant 




compared to placebo 








increase in overall survival compared to the placebo- 


• 


The observed increase in overall survival 


30 


(19.6) 




control group (p <0.05). Which of the following 




is so large that there is less than a 5% 








statements is most correct? 




chance that placebo is equal to the 












treatment 










• 


If the treatment does not actually increase 


41 


(26. 8)*^ 








survival, then the chance of obtaining the 












observed (or even greater) increase in 












survival is less than 5%. 












1 do not know and do not wish to guess 


24 


M 5 7^ 


CI vignette 


You read in a medical paper that the RR for a disease 


• 


95% of those exposed to the risk factor 


44 


(28.8) 




in individuals who are exposed to a risk factor is 1 .64 




naa km oeiween i.iu ana ii.4o 








(95% CI =1.10-2.45) compared to otherwise similar 


• 


1 here is a 95% likelihood that the true KK 


51 


(33.3)" 




individuals who are not exposed to this risk factor. 




falls between 1.10 and 2.45 








Which of the following statements is most correct? 


• 


Approximately 50% of individuals 


9 


(5.9) 








exposed to the risk factor had a RR of 










• 


1 .64 

1 do not know and do not wish to guess 


49 


(32) 


Correlation 


You read in a medical paper that there was a highly 


• 


There was a strong correlation between 


97 


(63.4) 


coefficient 


statistically significant positive correlation between 




atrial natriuretic peptide blood levels and 






vignette 


atrial natriuretic peptide blood levels and treatment 




treatment outcomes 








outcomes {r = 0.29, p < 0.001). Which of the following 


• 


The correlation between atrial natriuretic 


23 


(15)'' 




statements is most correct? 




peptide blood levels and treatment 












outcomes was weak 










• 


No correlation between atrial natriuretic 


4 


(2.6) 








peptide blood levels and treatment 












outcomes was detected 










• 


1 do not know and do not wish to guess 


29 


(19) 


Sensitivity 


An article is describing the effectiveness of a novel 


• 


The ratio of true positives to the number 


79 


(51.6)" 


vignette 


diagnostic technique for the detection of a disease. 




of subjects who have the disease 








The sensitivity of the technique is defined as; 


• 


The ratio of true positives to the number 


54 


(35.3) 



of subjects who had positive test results 
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Table 2 (Continued) 



Response 



Scenario description 


Multiple choices (select only one) 


frequencies'^ (%) 


• 


The ratio of false positives to the number 


8 (5.2) 




of subjects who had positive test results 




• 


1 do not know and do not wish to guess 


12 (7.8) 


RR vignette You read in a medical paper that a new screening test . 


The data indicate that the new screening 


49 (32) 


for a disease reduces by 25% the risk of death from 


test may achieve greater reduction of the 




the disease in high-risk individuals compared to those 


risk of death from the disease compared 




who do not undergo the screening test (RR = 0.75). On 


to the old screening test 




the other hand, you are also aware that 12 out of 1 ,000 . 


The data indicate that the old screening 


43 (28.1)" 


high-risk individuals who undergo an older screening 


test may achieve greater reduction of the 




test will die from the disease compared to 20 out of 


risk of death from the disease compared 




1 ,000 high-risk individuals who are not screened with 


to the new screening test 




any method. Which of the following statements is • 


The data indicate that both screening 


10 (6.5) 


most correct? 


tests have similar efficacy in reducing the 






risk of death from the disease 




• 


1 do not know and do not wish to guess 


51 (33.3) 



^Percentages in parentheses represent the rate percent to the study's 153 respondents. 
"^Correct answer. 

SD, standard deviation; SEM, standard error of the mean; CI, confidence interval; RR, relative risk. 



Table 3. Cognitive biases questions 



Scenario description 



Multiple choices (select only one) 



Response frequencies'^ (%) 



Gambler's Approximately 10 patients with hemoptysis 
fallacy per month attend a primary care center, 
vignette Epidemiological data from this center 
indicate that approximately 1 out of 1 0 
hemoptysis patients will have TB. Near the 
end of the month, nine patients with 
hemoptysis have attended the center and 
all were TB-negative. Which of the following 
statements about the next patient with 
hemoptysis who will attend the primary 
center before the end of the month is most 
correct? 

Conjunction John is a 42 years old, married office worker 
fallacy with a BMI of 35.6. He complains of angina- 
vignette like chest pain. His father had CAD and died 
from myocardial infarction 7 years ago. 
John's brother also suffers from CAD. 
Based on the above data, rank the following 
statements from one to six where one is the 
most probable and six is the least probable 
statement: 



It is almost 100% likely that the patient 
will have TB 

The patient's chances of having TB will 

be approximately 50% 

The patient's chances of having TB will 

be approximately 10% 

The patient's likelihood of having TB will 

be increased compared to the previous 

9 patients with hemoptysis but more 

data will be required to calculate it 

No conclusions about the patient can be 

derived from the scenario's data 

John has depression 

John is suffering from osteosarcoma and 

pneumonia 

John has CAD 

John has Huntington's disease 

John has pre-diabetes 

John has CAD and Huntington's disease 



4 (2.6) 

7 (4.6) 
83 (54.2)" 
30 (19.6) 

29 (19) 

48 (31 .4) respondents ranked 
the likelihood that John has 
Huntington's disease higher 
compared to the likelihood 
that he has both CAD and 
Huntington's disease 



^Percentages in parentheses represent the rate percent to the study's 153 respondents. 
"Correct answer. 

BMI, body mass index; TB, tuberculosis; CAD, coronary artery disease. 
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calculated. Each correct response counted one point while 
no points were given for incorrect answers. Furthermore, 
a 'total biostatistics familiarity' score was calculated by 
adding all familiarity ratings. 

Statistical analysis 

Data analysis was performed using R (Foundation for 
Statistical Computing, Vienna, Austria) (32). Variables 
were maintained as continuous or categorical according 
to their original form in the questionnaire. A non-normal 
distribution was assumed for all continuous variables 
as indicated by the Kolmogorov-Smirnov test. Missing 
values were counted as incorrect responses and added in 
the 'I do not know and do not wish to guess' response 
selection (Table 2). Median differences between more 
than two groups were evaluated using the Kruskal-Wallis 
one-way analysis of variance by ranks. The Spearman 
rank correlation coefficient was used to determine the 
correlation between two continuous variables. Median 
differences between two groups were analyzed using the 
Mann- Whitney U test for two non-paired data. Catego- 
rical variables were compared using Pearson's Chi-square 
or Fisher's exact tests where appropriate. The internal 
consistency of the 'total biostatistics familiarity' and 
the 'total biostatistics knowledge' scores was assessed 
by examining the Cronbach's a coefficient. Cronbach's 
a>0.6 were considered to be acceptable (33). Bivariate 
analyses were performed to identify individual resident 
characteristics that may be associated with total bio- 
statistics knowledge score, total biostatistics familiarity 
score, or performance on the cognitive biases questions. 
Candidate factors included age, gender, hospital, resi- 
dency field, residency year, residency program type, 
possession of other advanced degrees, education abroad, 
past biostatistics training, and total biostatistics famil- 
iarity score. Variables with p < 0.05 on bivariate compar- 
isons were further included in multivariate regression 
analyses. Multivariate analysis was performed with linear 
regression analyses as well as binary logistic regression 
tests using the dichotomous coding of responses (correct/ 
incorrect) as the dependent variable. To adjust for multi- 
ple pairwise comparisons, p <0.0l was considered statis- 
tically significant. 

Results 

The survey response rate was 76.5% (153/200). Table 1 
details participant demographic profile, education, and 
specialty choice. The majority of residents (81.7%) re- 
ported prior attendance of biostatistics courses, which 
mainly occurred during medical school. 

Responses to each of the individual biostatistics knowl- 
edge and cognitive bias questions are presented in Tables 2 
and 3, respectively. The proportion of correct answers to 
each biostatistic knowledge question ranged from 1 5% in 
the scenario concerning correlation coefficients to 51.6%i 



in the diagnostic sensitivity question. The most popular 
incorrect answer (selected by 63.4% of respondents) in 
the correlation coefficient question stated that the correla- 
tion between the two variables was strong. However, the 
correlation coefficient provided in the vignette (r=0.29) 
was actually low. Only two participants (1.3%) answered 
all biostatistics knowledge questions correctly while 29 
residents (19'/o) gave incorrect answers to all seven ques- 
tions. Reliability analysis of the 'total biostatistics 
familiarity' and the 'total biostatistics knowledge' scores 
showed acceptable a values of 0.904 and 0.648, respec- 
tively. The mean + SD of the 'total biostatistics knowl- 
edge' score was 2.1 + 1.8 (range 0-7). No significant 
association was found between residency field and total 
biostatistics knowledge score (p =0.066). On the other 
hand, bivariate analyses indicated a significant associa- 
tion of total biostatistics knowledge score with residents' 
age (r= -0.166, p =0.041), training year (r = -0.205, 
/)= 0.011), total biostatistics familiarity score (?• = 0.333, 
/)<0.001), and education abroad (p =0.004). Multivari- 
ate analysis was further performed in order to estimate 
the independent effects of these factors on biostatistics 
knowledge (Table 4). Residents' perceived familiarity 
with biostatistics was associated with higher total knowl- 
edge scores (p < 0.001) while previous education outside 
Greece was associated with lower total knowledge scores 
(p =0.005). 

Perceived familiarity ratings of biostatistical concepts 
by medical residents are listed in Table 5. Thirty-one 
residents (20.3'/o) gave scores <4 on all familiarity ratings. 
Notably, < 25% of residents reported having above aver- 
age knowledge (familiarity rating > 4) of correlation 
coefficients. The mean familiarity score for this statistical 
concept was 2.9+1.9 (mean + SD) which was significantly 
lower (p <0.01) compared to all other famiharity ratings. 
Bivariate analyses showed that residents with higher self- 
reported familiarity with SD (p < 0.001), SEM (p = 0.001), 



Table 4. Multivariate association (multiple linear regression) 
between resident variables and total biostatistics knowledge 
score 





Regression coefficient^ 


P 


Age (years) 


0.046 (-0.059 to 0.150) 


0.389 


Year of residency 


-0.227 (-0.429 to -0.026) 


0.027 


Education abroad'^ 


-0.846 (-1.426 to -0.266) 


0.005"= 


Total biostatistics 


0.047 (0.026 to 0.068) 


< 0.001 



familiarity score 



^Numbers in parentlieses represent the 95% confidence intervals 
(CI). 

'"No education abroad' is used as the reference category. 
"^Statistically significant association set at p<0.01 to adjust for 
multiple pairwise comparisons. 
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Table 5. Perceived familiarity rating by medical residents for the biostatistical concepts of SD, SEM, p-values, CI, correlation 
coefficients, RR, sensitivity, and PPV 



Familiarity rating^-'' 1 (none) {%) 2 (%) 3 (%) 4 (%) 5 (%) 6 (%) 7 (excelient) (%) 



SD 


24 (15.7) 


7 (4.6) 


26 (17) 


11 (7.2) 


28 (18.3) 


26 (17) 


31 (20.3) 


SEM 


37 (24.2) 


10 (6.5) 


23 (15) 


24 (15.7) 


24 (15.7) 


19 (12.4) 


16 (10.5) 


P 


34 (22.2) 


13 (8.5) 


16 (10.5) 


21 (13.7) 


19 (12.4) 


29 (19) 


21 (13.7) 


CI 


46 (30.1) 


16 (10.5 


23 (15) 


19 (12.4) 


24 (15.7) 


14 (9.2) 


11 (7.2) 


Correiation coefficients 


54 (35.3) 


21 (13.7) 


30 (19.6) 


14 (9.2) 


11 (7.2) 


13 (8.5) 


10 (6.5) 


RR 


33 (21.6) 


11 (7.2) 


26 (17) 


23 (15) 


23 (15) 


36 (23.5) 


1 (0.7) 


Sensitivity 


14 (9.2) 


7 (4.6) 


14 (9.2) 


16 (10.5) 


14 (9.2) 


36 (23.5) 


52 (34) 


PPV 


20 (13.1) 


12 (7.8) 


15 (9.8) 


24 (15.7) 


22 (14.4) 


30 (19.6) 


30 (19.6) 



"Percentages in parentheses represent the rate percent to the study's 1 53 respondents. 
"^Percentages in parentheses do not add up to 100% due to rounding. 

SD, standard deviation; SEM, standard error of the mean; CI, confidence intervals; RR, relative risk; PPV, positive predictive value. 



/>values (p =0.009), CI (/'<0.001), and correlation co- 
efficients (/)< 0.001) performed better on the respec- 
tive knowledge question. Also, there was no significant 
association between self-reported understanding of the 
concepts ofRR(/)=0.134)and sensitivity (/? = 0 . 5 7 1 ) with 
the respective knowledge question. The mean + SD of the 
'total biostatistics familiarity' score was 31.6 + 12.4 (range 
8-55). In bivariate analyses, total biostatistics familiarity 
was significantly associated with past training in bio- 
statistics (/7 =0.006) and possession of other advanced 
degrees (/?< 0.001). Conversely, residency field was not 
significantly associated with total biostatistics familiarity 
(p = 0.236). A linear regression model was fitted with 'total 
biostatistics familiarity' score as the dependent variable 
and past biostatistics training and possession of other 
advanced degrees as the independent variables (Table 6). 
Residents who have previously attended biostatistics 
courses tended to report higher familiarity scores al- 
though this effect did not reach statistical significance 
(p>0.01). Residents with other advanced degrees were 
significantly more likely to report higher familiarity with 
statistical concepts {p < 0.001). 



Table 6. Multivariate association (multiple linear regression) 
between resident variables and total biostatistics familiarity 
score 





Regression coefficient'^ 


P 


Biostatistics training'^ 


6.054 (1.205 to 10.904) 


0.015 


Other graduate degrees'^ 


11.349 (5.382 to 17.316) 


< 0.001"^ 



''Numbers in parentheses represent the 95% confidence intervals 
(CI). 

'''No prior biostatistics training' is used as the reference category. 
"^'No other graduate degrees' is used as the reference category, 
''statistically significant association set at p <0.01 to adjust for 
multiple pairwise comparisons. 



As shown in Table 3, approximately half of the 
respondents (54.2%) answered the gambler's fallacy quiz 
correctly. Furthermore, less than one third of residents 
surveyed (31.4'/o) judged the probability that John has 
Huntington's disease to be greater than the probability 
that John has both CAD and Huntington's disease. 
Preliminary bivariate analysis of factors that may affect 
resident performance on the gambler's fallacy problem 
was followed by multivariate logistic regression which 
showed that residents with higher total biostatistics 
scores were more likely to answer that quiz correctly 
(odds ratio = 1.38, 95% CI 1.12-1.70; 7? =0.003). Also, 
none of the investigated individual resident factors were 
found to be significantly associated with performance on 
the conjunction fallacy quiz. 

Discussion 

The present multicenter survey of biostatistics perfor- 
mance and biases demonstrated that a considerable pro- 
portion of medical residents are prone to errors and biases 
that undermine judgement and decision making. The fact 
that, on average, only two out of seven biostatics knowl- 
edge questions were answered correctly raises serious 
concerns. This significant lack of biostatistics aptitude 
was observed despite the fact that the majority of 
participants have attended undergraduate biostatistical 
training during medical school. It should be noted, 
however, that most residents did not reinforce these 
courses during residency training. Indeed, the biostati- 
stics knowledge score tended to decline with progression 
through the years of residency training although this 
observation did not reach statistical significance (Table 4). 
Furthermore, we did not observe a significant perfor- 
mance difference between residents who have previously 
received biostatistics education with those that have never 
received such training at any point in their career. Taken 
together, these data suggest that biostatistics skills may 
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atrophy with progression through medical training and 
further effort should be expended to maintain knowledge 
in that sphere during residency. 

The majority of residents were unable to properly 
combine significance level information with correlation 
coefficient data (Table 2). This mistake was observed 
even in respondents who have selected the correct defini- 
tion of the ;7-value in the relevant question. Educators 
may therefore need to reevaluate how these concepts are 
taught and represented in a way that medical residents can 
understand and use in rational, evidence-based decisions. 

Further analysis of the effects of individual resident 
variables (Table 4) revealed that respondents (all Greek 
citizens) who have received medical education outside 
Greece, predominantly taking place during undergraduate 
medical education in international medical schools, had 
lower biostatistics knowledge scores compared to resi- 
dents who have never trained abroad. This performance 
difference may be partly due to variability in medical 
school experiences obtained by Greek citizens abroad 
but it may also reflect differences in ability and effort. 
Further research will be required to elucidate the potential 
performance differences between graduates of Greek and 
international medical schools particularly with regards to 
educational quality, clinical outcomes and quality of care. 
It should also be noted that other resident characteristics, 
including age, gender, advanced Graduate training, speci- 
alty choice, hospital and residency program type did 
not affect the biostatistics knowledge score of residents 
indicating that the insufficient biostatistical literacy re- 
ported in the present study is similar across a broad range 
of medical residents in Greece. Furthermore, biostatistical 
performance was homogenous among residents training 
in different specialty programs. To our knowledge, the 
present study is the first to investigate and directly 
compare the biostatistics knowledge between different 
resident specialty fields. Recent studies have assessed the 
biostatistical competence of internal medicine, emergency 
medicine, and obstetrics-gynecology residents (15-17). 
Contrary to the present study, Windish et al. reported that 
prior biostatistics training, gender and advanced degrees 
were predictive of internal medicine residents' statistics 
knowledge (15). Also, a Danish study mainly focusing on 
junior hospital doctors did not find significantly higher 
statistics performance in physicians with previous biosta- 
tistics education or interest in research (10). These reports 
used different sets of questions compared to this study, 
thus hindering direct comparisons. 

Approximately one out of five medical residents 
acknowledged below average familiarity with eight bios- 
tatistical concepts that are routinely encountered in the 
medical literature (12, 14). This lack of comfort was sub- 
stantiated by the similar percentage of residents who 
answered all biostatistics questions erroneously. Residents 



considered themselves the least familiar with the concept 
of correlation coefficient. Indeed, their actual perfor- 
mance in the correlation coefficient quiz was clearly lower 
compared to all other tests. The independent association 
between self-reported biostatistics familiarity and knowl- 
edge score was further verified in a multivariate model 
(Table 4). It is of note that residents with advanced 
education through a master's or PhD degree were sig- 
nificantly more likely to report higher familiarity with 
biostatistical concepts although this effect did not trans- 
late into substantial performance improvement on the 
knowledge questions. Previous education in biostatistics 
also tended to independently increase familiarity rankings 
although this association did not reach statistical signifi- 
cance. These observations indicate that although residents 
with advanced training and prior statistical education 
are more likely to believe that they are comfortable with 
statistics compared to other medical residents, they ex- 
perience similar difficulties in interpreting research statis- 
tics. Consequently, formalized systematic teaching of 
biostatistics during residency will be required to ensure 
that residents are proficient in understanding and com- 
municating statistical information. 

To the best of our knowledge, the present study is the 
first to investigate the susceptibility of physicians to the 
gambler's fallacy and the conjunction fallacy in medi- 
cal settings. A recent survey in Germany reported that 
approximately 60% of the general population selected the 
correct answer in a coin toss gambler's fallacy test (27). 
The pervasiveness of this bias among our study population 
of medical residents is alarming, especially given the fact 
that the bias was investigated using a problem within their 
domain of expertise. Residents who felt that the patient 
has increased chances of having tuberculosis are likely to 
overlook other important causes of hemoptysis. Notably, 
residents with higher biostatistics knowledge scores were 
less prone to the gambler's fallacy. Therefore, continuing 
medical education programs focusing in statistical con- 
cepts and reasoning may mitigate the probability judg- 
ment biases of residents (34, 35). Furthermore, specific 
instruction aimed at increasing awareness of the gambler's 
fallacy may reduce residents' susceptibility to this bias 
(35-37). 

Approximately 80-90% of respondents generally vio- 
late the conjunction rule according to previous reports 
using various presentations of the 'Linda' problem 
(29, 38, 39). In the present study, we found a slightly 
lower percentage (68.6%) of violations of class-inclusion 
in medical residents using a clinically based problem. 
This percentage was found to be independent of resi- 
dents' individual characteristics including age, gender, 
residency year, past biostatistics training, performance on 
the biostatistics tests and possession of other advanced 
degrees. A number of authors have argued that linguistic 
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and pragmatic features in conjunction fallacy scenarios 
may lead respondents to alternative evaluations (38, 40). 
For example, if residents interpret the statement 'John 
has CAD' to additionally imply that 'John does not 
have Huntington's disease' then ranking this statement 
as more probable than 'John has CAD and Huntington's 
disease' would not be considered a conjunction fallacy. In 
addition, similarly to the English and German languages, 
the word 'probable' is polysemous in Greek and residents 
faced with multiple possible interpretations may infer a 
non-mathematical meaning of probability. 

A further interpretation of the high number of con- 
junction violations found in the present study may be that 
residents apply different inferential rules and heuristic 
procedures resulting in answers that appear mathemati- 
cally incorrect (41). Physicians use a repertoire of clinically 
relevant assumptions to arrive at a differential diagnosis 
based on the patient's history, symptoms and physical 
examination. These diagnostic norms tend to focus on the 
most relevant information and assign that information its 
appropriate weight taking into account various clinical 
goals and expectations. This strategy may indeed disregard 
syntactical systems such as the conjunction rule. However, 
medical decision making is not dependent solely on the 
probabihty of an event (e.g., a disease). More often, 
clinicians focus on calculating the probability times pay- 
off or cost when constructing a differential diagnosis list. 
Although this approach can lead physicians to rank the 
constituent as more probable than the conjunction it 
may also reflect an effective and intelUgent way to tackle 
clinical uncertainty. During pretesting of our conjunction 
fallacy vignette, residents were specifically made aware of 
the mathematical nature of the test and the class-inclusion 
violation of ranking the probability of CAD and Hun- 
tington's disease as more probably than Huntington's 
disease. However, they still insisted that paying attention 
to the gist that the patient had multiple risk factors for 
CAD was far more reasonable than precise mathematical 
rankings. 

Limitations 

The cross-sectional design of the present study prevented 
determination of causality. Furthermore, in order to fully 
protect residents' anonymity, we were unable to collect any 
further data on non-respondents. Although the response 
rate was significantly higher than what is typical in surveys 
of physicians (42), we cannot exclude the possibility that 
residents who felt less comfortable with statistical concepts 
may have been less willing to complete and return the 
questionnaire. The present survey was purposely kept 
brief in order to achieve maximum participation. We 
thus limited the assessment of residents' biostatistics 
knowledge to seven biostatistical concepts (Table 2) which 
however represent some of the most commonly used 
statistical methods found in the medical literature today. 



Conclusions 

The measured biostatistics performance in our survey 
indicates that an alarmingly large number of medical 
residents are unable to correctly interpret crucial statistical 
concepts that are commonly found in the medical litera- 
ture. These difficulties were pervasive, extending even to 
residents with prior biostatistics education and advanced 
graduate training. In addition, residents were found to 
be especially prone to the gambler's fallacy bias, which 
may undermine clinical judgment and medical decision 
making. Frequent violations of the conjunction rule in a 
clinical scenario were also observed and residents were 
uncomfortable with the mathematical explanation of the 
problem. The low performance of medical residents in the 
biostatistics knowledge tests and the gambler's fallacy 
question was independent of any prior biostatistics edu- 
cation, which mainly occurred during medical school. 
Therefore, in order to adequately develop and maintain 
the biostatistical reasoning of medical residents, educators 
may need to re-evaluate how such information is taught 
as well as emphasize and systematize the teaching of 
statistical concepts during residency. 
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